More Research Is Needed Before Hospital Quality Metrics Are Implemented

The American Heart Association/American Stroke Association have developed a Scientific Statement addressing the challenges of measuring and risk-adjusting ischemic stroke outcomes in order to compare quality of care across US hospitals.¹ At first glance this may not appear to be a compelling issue for practicing clinicians; however, quality and outcome data obtained from administrative claims are likely to become increasingly consequential as public reporting of hospital-level quality metrics and pay-for-performance initiatives are developed. The discussion and recommendations in this Statement are therefore of tremendous importance to patients, providers, payers, and health policy-makers.

Stroke is highly prevalent and is a leading cause of mortality and adult onset disability.² As a result, care for patients with stroke accounts for a meaningful portion of expenditures by the Centers for Medicare and Medicaid Services (CMS).^3,4 With the untenable growth in health care costs there have been concerted efforts by government and private payers to tie reimbursement to the quality of care received.⁵ Pay-for-performance programs provide a financial incentive for hospitals to deliver higher quality care. These programs are predicated upon an ability to accurately assess clinical outcomes. The CMS Hospital Inpatient Quality Reporting Program has proposed two stroke outcome measures to be utilized beginning in 2016: The 30-day risk-standardized stroke readmission rate and the 30-day risk-standardized stroke mortality rate. On the surface, these outcomes appear to be reasonable choices with high face validity. Both are objective and readily measured in existing administrative claims databases. Death has an obvious and profound impact on the patient and his/her family. Hospital readmission adds dramatically to healthcare cost and may be avoidable, at least in some instances.^6,7 Unfortunately, as outlined in the scientific statement, there are major methodological challenges which limit the utility of these outcome measures.

It is impossible to fairly compare outcomes across hospitals without adequately adjusting for differences in case mix and disease severity. Risk adjustment is necessary for all disease states, but in stroke there are unique challenges that limit the effectiveness of claims-based outcome measures. The authors of the Scientific Statement performed a literature review to determine factors associated with outcome from stroke and identified age, gender, comorbid conditions, vascular risk factors, and pre-stroke function as significant predictors. However, overwhelming each of these factors in determining outcome is the severity of the stroke symptoms at presentation to the hospital.⁸ It is intuitive that your final stroke outcome is strongly and directly linked to how severe it was at onset. Measures of stroke severity are not captured in administrative databases. Risk-adjustment models created by CMS, which do not account for stroke severity, have only modest discrimination (c-statistics 0.71 for 30-day mortality and 0.59 for 30-day readmission).^9,10 For 30-day mortality, a claims-based model that does not include a measure of stroke severity will misclassify a large proportion of hospitals.¹¹ The most commonly used measure to quantify stroke severity is the National Institutes of Health Stroke Scale (NIHSS) score. In the AHA/ASA Get With The Guidelines – Stroke database, where hospitals have by their inclusion already demonstrated an abiding interest in improving outcomes from stroke, the NIHSS was not documented in over 40% of patients.¹² In clinical practice, the availability of stroke severity measures is sure to be far less. As a result, adding these data to existing administrative datasets is not likely, at least in the short term.

Public reporting of hospital quality and pay for performance should lead to more informed healthcare consumers and improved quality of care.¹³ However, there are potentially serious unintended consequences to these initiatives if risk adjustment is inadequate, particularly if there are systematic differences in disease severity across hospitals. This is likely to be the case for stroke, since EMS policy in many states preferentially routes the most acutely ill patients to specialized stroke centers.¹⁴ Developing primary and comprehensive stroke centers is a labor- and resource-intensive endeavor. Inadequate risk adjustment could result in financial penalties for these hospitals, which could then result in hospitals avoiding the sickest stroke patients or electing not to pursue stroke center status. Similarly, referral centers and primary/comprehensive stroke centers could be less inclined to accept transfers of patients with a devastating stroke. These negative incentives could result in exacerbation in disparities for vulnerable populations, and the rapidly developing stroke systems of care would be undermined. Finally, basing these programs on mortality alone, without accounting for the functional status of survivors, could reduce the use of palliative care in the most severely injured patients. Given the extremely low quality of life associated with severe disability and the high costs of long- term care this would be harmful to the patient and to society.^15,16 As the statement points out, functional status is not uniformly captured in administrative, or even clinical, databases and there is currently no way to incorporate a measure of functional status into hospital-level quality metrics using claims.

There has been an impressive and heartening trend in reducing stroke mortality over the past 4 decades.¹⁷ While much of this is attributable to more aggressive management of vascular risk factors, there is likely a meaningful contribution from the development of specialized stroke centers. Stroke centers have lower mortality and utilize more acute stroke therapies, which reduce disability after stroke.^18-21 Appropriate and accurate measurement of hospital-level outcomes may allow for further optimization of stroke care systems by directing patients to the hospitals best equipped to care for them. However, public reporting and pay-for- performance programs should not be initiated until there is a better understanding of hospital-level outcomes and risk standardization models have been more carefully validated as the potential for negative consequences is too great. As the AHA Scientific Statement concludes, there is an urgent need for additional research in this area.